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Abstract: Web-enabled technology is now being applied on a large scale. In this paper we look at open access provision 
of teaching and learning leading to many users with varying patterns and motivations for use. This has provided us with a 
research challenge to find methods that help us understand and explain such initiatives. We describe ways to model the 
research and identify where pressures and contradictions can be found, drawing on a reflective view of our own practice 
in performing the research. Open educational resources are defined as technology-enabled educational resources that 
are openly available for consultation, use and adaptation by users for non-commercial purposes (UNESCO, 2002). 
OpenLearn is one of the largest of such initiatives and is committed to the provision of open educational resources for all. 
It is being developed by The Open University and is primarily sponsored by the William and Flora Hewlett Foundation. It 
provides users with over 4 200 hours of higher educational material drawn from Open University courses. Other learning 
tools such as discussion forums, video conferencing, and knowledge mapping software are also available to the user. In 
this paper we aim to introduce OpenLearn and outline some of the main research issues surrounding such an initiative. 
We seek to explore theoretical and practical approaches that can provide suitable tools for analysis. Activity theory is 
seen as a suitable approach for macro analysis and its use is illustrated in terms of the complexity of large scale 
research. Activity theory, besides informing research perspectives, can be turned in upon the research process itself 
allowing us to consider the challenges and context of the research. By using activity theory in this way and illustrating 
from a range of practical approaches we demonstrate and illustrate a useful research approach. 
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1. Introduction 

Open educational resources are defined as technology-enabled educational resources that are openly 
available for consultation, use and adaptation by users for non-commercial purposes (UNESCO, 2002). The 
internet has recently seen an increase in such initiatives the range including areas such as MIT 
OpenCourseWare, CORE (China Open Resource for Education), Wikipedia, OpenLearn and 
OpenCourseWare Universia. Key features of many of such initiatives include the provision of course content, 
the ability to adapt, use and develop content, the availability of social learning tools, and the introduction of 
other learning tools. The range of initiatives is illustrated by the membership of the OpenCourseWare 
Consortium ( http://www.ocwconsortium.org) which has members from a diverse number of countries. 

The impact of these resources and changes in the ability of individual users to access such educational 
materials is likely to impact significantly on how people learn. The confinement of knowledge to educational 
institutes is likely to be challenged. Alternatively, new power structures may arise as a result of ‘big players’ 
dominating the open resource markets. The question arises as to how these open resources will impact on 
the learning of individuals or groups of learners. Will these resources empower learners and how will 
learners’ experience change? Coupled with the advent of open resources there is also the development of 
new software and hardware tools such as software that supports social learning and mobile computing that 
may change the possible affordances of such technologies. For instance by combining wireless connectivity 
with mobility the ability to access information when wanted becomes more achievable. OpenLearn (Figure 1) 
is a major open content initiative funded by the Hewlett foundation and The Open University. The project was 
officially launched on 25 October 2006 and currently provides over 4 100 hours of university level material for 
personal study. The material used in the LearningSpace is currently derived from Open University course 
material. This has been quality assured and is currently in use or has been used within Open University 
validated courses. The conversion of this material into an appropriate provision for OpenLearn was 
conducted using an ‘integrity’ model where the content is kept as close as possible to the original source 
material with adjustments made relating to presentation on the web, rights issues, and reshaping for a wider 
audience. These units of study are located within an area of the OpenLearn site known as the 
LearningSpace. The LearningSpace also allows users to keep journals relating to their activity and to 
participate in forums relating to each of the units. 
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Another area of OpenLearn is known as the LabSpace. This space is envisaged as being more experimental 
and developmental in nature. It is possible for users to download existing content, edit, modify or add to it 
and upload it as a new version. Educational materials from the Open University’s archives are also made 
available within this space. These materials may be re-worked or re-versioned by users. Users are also able 
to upload their own course materials into the LabSpace. If the material is suitable it may then be transferred 
across from the LabSpace into the LearningSpace. The LabSpace contains a number of tools that might be 
used to help learning, or help a course developer. Compendium is a mapping tool that can be used to help in 
planning and structuring courses but can also be used for other applications such as mind mapping, concept 
mapping and the development of integrative diagrams for research. FlashMeeting (Figure 2) is a video 
conferencing tool that allows a number of participants to interact. It uses a low but useful video frame rate 
allowing a better audio reception over normal web connections. The OpenLearn servers can cope with up to 
100 different users simultaneously, or 50 meetings with two individuals. FlashMeeting is a one click 
videoconferencing tool and does not need to be installed as long as the free Adobe Flash Player runs on the 
web browser. The tool uses the ‘simplex’ or ‘push-to-talk’ audio tool allowing only one to be broadcasting 
sound at any one time which is ideal for interviews and multi-party broadcasting. The conferences are 
recorded remotely and may be viewed privately at a later date or made available to the world at large. The 
LabSpace also includes a chat tool, MSG, which is a text driven messenger. 

The roles of the LearningSpace and LabSpace and their inter-relationship are in a state of continual 
evolution. Recently, for instance Compendium has been additionally made available in the LearningSpace. 
Once a course has been uploaded to the LearningSpace the link to each course is relatively stable although 
at a later date a course may be replaced, updated or moved to the LabSpace. FlashMeetings can be booked 
at any time via the relevant link within the LabSpace. All FlashMeetings are permanently recorded and are 
either restricted to a group designated by the users, or can be made open to the public. Users can provide 
links directly to their recorded FlashMeeting without having to go the main OpenLearn site. A number of 
forums exist attached to the subject areas. These are readable for all and writeable for OpenLearn 
registrants. Some of the tools such as MSG (chat) are currently restricted to registrants only. 

A typical user might enter the site and decide to enrol on a unit. Each unit would take between and estimated 
3 to 15 hours of systematic study. At the time of writing there were 352 units in the LearningSpace of 
OpenLearn. Examples of units include; French: Ouverture, Knowledge Mapping, Understanding Operations 
Management, The Meaning of Home, and Earthquakes. Many users will dip into the units, reading some bits 
and skimming others. Others might focus on tool usage, social learning or using and drawing from materials 
for teaching purposes. 

The servicing of OpenLearn and its maintenance is funded by the project grant from Hewlett Packard and 
the Open University. One of the main themes of the research is how to sustain the project beyond the end of 
this funding. 

2. OpenLearn - The research challenge 

OpenLearn is a large project with a dedicated research team consisting of three individuals involved with 
research and evaluation. A significant proportion of the project budget has been set aside for research. 
Besides this dedicated research team others within the project have research interests including academics 
responsible for the transformation and development of content, and those involved with the research and 
development of software tools. The project as a whole is viewed in terms of an action research model where 
the results and impacts of research are fed back into project development. Within this paradigm there is also 
the challenge and the tension between academic research requiring high degrees of rigour and having 
underlying theoretical aims and that of applied research with the requirement of fast-feedback and relating 
more to the success of the site via such issues as marketing, usability studies, site design etc. 
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Figure 1 OpenLearn Home Page giving access to LearningSpace and LabSpace (viewable at 
http://www.open.ac.uk/openlearn ) 
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Figure 2: FlashMeeting Conference Interface - this resource is freely available in OpenLearn simply 
requiring a WebCam and a headset from users. (viewable at 
http://labspace.open.ac.uk/course/view.php?name=FM ) 

In addition to team members and their research interests other sources of research input have been used. 
This has included the formal employment of university expertise for usability studies and the help of PhD 
students. Non-formal sources of research input have included voluntary interactions with the OpenLearn 
team and academic input from the wider community. This community includes those with expertise in 
eLearning, many within the university, who may share formally or informally in the development of the 
project. Further afield there is the impact and influence of the education research community, especially in 
relation to eLearning, and the OpenCourseWare Consortium that provides a structured working environment 
and opportunity for the coming together of various open content providers. 


Researching such a large project involves examining several different areas, each of which presents its own 
set of challenges. The four main strands of research include; teaching with OpenLearn, the users 
experience, project development, and sustainability. To this we can add a meta-layer; the challenge of 
researching open content. This paper is chiefly concerned with this objective and aims to illustrate our own 
problems and dilemmas in conducting such research and by offering our insights into models of research 
provide a guide for others in research techniques. 


An idealised approach to the research has been to: 

1. identify and develop theoretical frameworks for analysis at macro and micro levels 

2. find tools and ways of mapping and talking about our research 
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3. develop and find appropriate methodologies to enable us to collect and process research findings 

This list may seem very logical and structured as presented here but the evolution of the framework, as we 
learn within an action research paradigm is more haphazard and iterative in nature. A research technique is 
seen not as something developed in advance but rather as something in the process of development with 
continual trials, implementations and reworking. OpenLearn itself is a continually evolving construct and this 
presents an additional challenge to research. The three approaches listed above will be described in more 
detail with specific reference to research within the project and also to researching users’ experience. 

2.1 Developing theoretical frameworks of analysis at the macro and micro levels 

2.1.1 Action research 

One possible criticism of academic research is that the impacts of such research often have little effect on 
practice. A possible reason for this may be the divide that exists between the world of academia and the 
world of work. The process of academic research can be very slow with the major outputs often consisting of 
writings in journals for an academic audience. Research is often conducted from ‘afar’ that is, it is separate 
from the object of research. An advantage of this is that research is more likely to be independent if not 
connected to the object of the research. This independence and objectivity is unlikely to be untainted in that 
research and researchers are embedded within research paradigms, personal social-cultural influences, and 
the influence of the grant holders who partially or wholly shape the research questions. The principles of 
action research call for a research process that involves change within that which is researched (Greenwood 
et al 2006, Somekh 2006). In a sense it is more of an experimental ‘trial and error’ process in that it is 
iterative, ongoing and affects change in practice. It can therefore be seen as a process of reflection and 
practice, often referred to as praxis. In order to affect action research it is necessary to 

1. involve more of the organisation than simply the dedicated researchers, 

2. to integrate the results of the research into decision making at managerial levels. 


Dangers exist however when moving towards a culture of ‘self-development’ where Action Research is seen 
as an efficiency tool as opposed to its more idealised aims of democratisation, development and 
empowerment of workers (Greenwood et al, 2006). There are also dangers when research is taken out of 
the hands of research savvy practitioners and placed in those of research novices. Hence there may be 
many models of action research adopted according to one’s perspective. Another key issue of action 
research is the ‘social-technical’ view which sees the successful development of any organisation being an 
integration of the right social and developmental environment with the use of appropriate tools. For example, 
the use of tools for doing research and for enhancing interpersonal communication within the research 
community and others in the organisation is part of praxis resulting from the research itself. Action research 
can provide us with a framework of research at the level of OpenLearn as an organisation but also as a 
framework of reflection and practice within the Research Unit. In this case we see this as a way of 
developing ourselves as individuals and as a team allowing an exploration of ways of working and knowing. 
Somekh (2006: 7) says: 

‘The self of the researcher can best be understood as intermeshed with others through webs of 

interpersonal and professional relationships that co-construct the researcher’s identity’ 

In this sense action research is about both personal and professional development. 

2.1.2 Activity theory as a way of modelling macro behaviour 

OpenLearn represents one of the largest educational interventions on the Internet and as such the 
opportunity exists to understand how this operates and develops at a macro level. Possible contenders for 
analysis include activity theory and actor network theory which allow potential ways of understanding macro¬ 
behaviour. Actor Network Theory (ANT) focuses on identifying the various actors in a social organisations 
and examining the relationship between these actors (Latour, 2005). Activity theory focuses on action as it is 
mediated by tools within a socio-cultural context (Cole and Engestrom, 2003). It was used as an analytical 
framework in this instance because of its educational applications including learning in organisations and 
that ANT was felt to be less clearly structured as an analytical tool. 

The foundation for Activity Theory comes from the Vygotskian view that all action is mediated by tools 
whether these be external or internal, concrete or psychological (Vygotsky, 1980). This has been developed 
into concepts such as ‘person plus’ and cognition as a distributed activity located within a social group and 
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the tools that they use (Perkins, 1993). Leont’ev, a prodigy of Vygotsky, explored the way in which this could 
be applied through emphasising the activity as the main unit of analysis (Kaptelinin and Nardi, 2006). 
Engestrom extended the framework and the subject-tools-object model to take into account aspects of the 
context within which such action was taken (Cole and Engestrom, 1993). He represented the inter¬ 
relationships between these contextual elements within a triangular structure each node representing some 
aspect of interaction. The additional contextual nodes that he added were ‘rules’, ‘community’ and ‘division of 
labour’ (Cole and Engestrom 1993, Kaptelinin and Nardi 2006). This framework was adopted as a practical 
tool of analysis since it could be applied to view OpenLearn from any number of different perspectives. 
These different perspectives could then be contrasted, reflected upon, or pushed against each other to force 
the identification of characteristics within each perspective and various ‘contradictions’ that existed between 
such perspectives. 

Figure 3 demonstrates the use activity theory as a way of viewing the Research aspects of the OpenLearn. 
Researching such a complex and large educational initiative provides many opportunities and areas for 
potential study and often these are driven by project aims. These aims can be envisaged as being part of the 
rules in which the research is located and represent rules embedded in project design. Other rules are 
external to the project and include guidelines for general social research. These deal with issues such as 
research ethics which can sometimes create tensions in terms of the need for fast feedback and the drives 
for ‘interesting stories’ that may come from other parts of the project such as marketing, or management. 
‘Rules’ may also relate to perceptions of individuals within the team (i.e. not formally held or shared rules) 
and relate to theoretical perceptions and opinions on the nature of good educational practice. Much research 
demands a certain standard of rigour (lower risk of error) which might create contradictions with the need for 
quick feedback (higher risk of error) to help move the implementation process forward. This contradiction 
highlights a general problem of the slowness of academic research to reach and inform its intended 
audience. By identifying and recognising this contradiction ways can be investigated for disseminating 
research internally in order to quickly feedback into the implementation and adaptation processes. 

When examining Figure 3 contradictions maybe analysed within the structure itself e.g. between the 
research interests, motivations, and perceived views of the team players, between individuals and rules, 
about the essence and nature of research itself, about the choice of methods to monitor the learning 
effectiveness of OpenLearn. Contradictions can also be viewed of as occurring across different perspectives. 
For example a contradiction may exist between the need for neutrality and a critical approach of the 
researchers within the research perspective and the need for promotion and publicity within a marketing 
perspective that is directed towards gaining the attention of users. 

2.1.3 Activity theory and action research 

It is clear that Action Research and Activity Theory can be used effectively together. As Somekh (2006: 22) 
says when talking about Action Research, 

‘...activity theory is particularly helpful because it gives priority to collaborative decision making on the 

basis of sharing knowledge about identified ‘contradictions’. 

The socio-technical aspects of action theory can be related to aspects of tool mediation and the development 
of community. Activity Theory can be used to identify contradictions and these can be used to implement 
change. Such change might create new contradictions but through the iterative process of action research 
improvements can be made. A multi-perspective approach can be used to inform those working within 
OpenLearn to affect change. The challenge of how this dissemination occurs is something that needs to be 
addressed. 
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Research Methods 
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Figure 3: Using activity theory to represent the research perspective within OpenLearn 

2.2 Tools for mapping and talking about research 

Taking into account the socio-technical aspects of Action Research and the central role of tools within 
Activity Theory the development of various tools for thinking, analysing and describing is a key part of our 
research process. With such an array of available media there are difficulties in identifying the most suitable 
way or ways for research discussion and dissemination. The development of a research community that 
allows the findings of OpenLearn to be discussed and disseminated can be seen as a good area for the 
application of action research. It can mean experimenting with different tools and the construction of different 
types of space. In discussing and disseminating research findings a number of technological options present 
themselves as possible platforms e.g. blogs, wikis, editable web sites and facilities which can be found in the 
LabSpace. Within the LabSpace there is the opportunity to create discussion forums around research issues. 
These forums can be opened up to anyone. Also we can provide our own research space within the 
LabSpace. Individual and collective blogs can also reveal insights into research and development of 
OpenLearn and open content as a whole. Linking and exploring blogs can be of value as a research activity. 
An example of a blogged resource open to all is that of the ‘OpenLearn2007’ (2007) conference held at the 
Open University, Milton Keynes. Each of the presentations was informally blogged by attendees and the 
blogs made available to the world at large. 

Compendium is an example of a software tool available within OpenLearn that can allow the development of 
concept maps, integrative research diagrams and help to structurally organise and develop courses. As a 
tool within OpenLearn it is still undergoing development. Any type of mapping allows the user to present in a 
way that combines text and graphics in a visual dynamic that can represent various structures, concepts and 
their relationships. They thus exist as a tool allowing users to reach beyond the limits of the mind in terms of 
its cognitive load. Although compendium has some limits compared with pen and paper it has affordances in 
that it is editable, re-mouldable, non-linear, allows multidimensionality, nesting and layering, allows links to 
other technical resources such as web pages, documents, images etc. It can also be shared dynamically 
between teams and individuals. Such a tool can allow us to model research complexity and represent 
discussion around theoretical issues. 

2.3 Developing appropriate methodologies 

Researching OpenLearn is a complex phenomenon and various tensions exist internal to the research 
perspective in such issues as: 

■ tensions between pedagogical paradigms 

■ intrusiveness of research methods 

■ nature of sampling and reaching hard to reach groups 

■ speed of research feedback 

■ choosing suitable analytical techniques 

■ dissemination methods 
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■ nature of research tools used 

■ dealing with a non-homogeneous user base 

As an example we can look at one of these issues in more depth. One of these is the fact the user base 
appears to be very heterogeneous and heavily skewed in terms of time spent on the site toward the low 
user. This pyramidal structure in terms of the time spent visiting a site is probably a common occurrence in 
many web sites although comparative studies are difficult because open content sites may differ greatly in 
form and function. Every website is in a sense competing against a large number of other sites in terms of 
grabbing a person’s attention and part of the decision that people make in spending time on a site is their 
initial perception of the site’s value. This can very much depend on how the home page of the site is 
presented and whether the site gives a clear indication of what kind of content or activity that it might contain. 

A major challenge is to find out about the user experience. Questions relating to this challenge include: 

1. how does the use of OpenLearn fit into the wider context of the user’s formal and informal learning 
context? 

2. are users learning from OpenLearn? If so how are they learning? 

3. are users engaging and using the tools? If so what are they using the tools for? 

Although there have been over a million discrete visitors to date the median estimated dwell time is relatively 
short with a heavily skewed distribution. For example about 50% of registered users have spent less than 30 
minutes on the site in total although some 10% have spend more than four hours on the site in total. Does 
this mean that we should assume that the users with low dwell times are simply browsing and not learning 
much? Or might they be picking up small chunks or bits of information? The challenge of finding out about 
the ‘fleeting’ user is different from that of finding out about the more substantial user. Distinctions will also 
exist in terms of types of users. Users are able to freely re-use the material and republish under the 
commons license although it is a small minority who may need or want to do so. A teacher may draw from 
OpenLearn content to use in their teaching. Teachers and academics may wish to edit and change content 
which they are able to do in the LabSpace. Others may be interested in joining in the OpenContent research 
participatory groups or forums to discuss the research and development of open content and open 
educational resources. Learners may primarily study or use content but others may engage with social 
learning or using OpenLearn tools. Part of the challenge is to identify the types of user. 

One possible means of tracing the user’s experience is by using website logs. Generally one can infer 
whether a user was browsing, skimming, downloading or printing content, or systematically studying or 
reading parts of a unit. This however does not tell us about what the user is learning and with OpenLearn it is 
not possible to use pre- and post-study tests since learners will often not be studying a unit as a whole. In 
this instance thinking of the units as courses is inappropriate since this implies a journey from a starting point 
to an end point, and an externally structured pathway through the material often with some form of 
assessment. Identifying learning therefore depends on the unique experiences of users and needs to be 
process orientated. In order to get at the experience of users and the process of learning qualitative studies 
can provide a rich picture and thick description of users’ experience. There are also several technological 
tools that can help the researcher in this process although there is always the problem of the degree of 
intrusiveness in any research exercise. 

One method of examining whether learning is taking place is by in situ observation and making inferences 
from user activity. Getting users to think aloud and to record their thoughts can help in this although there are 
disadvantages to this technique. Another is by using interviews where a user’s learning experience can be 
examined. Simple questions such as ‘What have you learned?’ or ‘What have you found out that you didn’t 
know before?’ can act as the basis for more probing questions perhaps relating to a range of skills within 
Bloom’s taxonomy (Bloom, M. et al 1956). A problem of such interviews is that they can become an 
additional form of teaching in that by causing the participant to recall or reconstruct their experience one is 
actually changing and reshaping it. Thus the research is adding to the learning experience. 

Remote monitoring can allow a clearer insight into the actual live experience of the learner. This can be 
achieved in a number of ways. One such method has been co-developed with a volunteer who has been 
examining how to change and modify unit content within the LabSpace. This involves downloading pre¬ 
existing content, modifying and/or adding to it, uploading it back into the LabSpace as a new version. The 
research process began with communication using email and a FlashMeeting interview (a free video 
conferencing made available within the LabSpace) where a fairly open ended task was set up. This involved 
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downloading one of the units from the LabSpace and making adjustments to the unit and adding to it i.e. ‘re¬ 
versioning’ the unit. 

Jane, the volunteer, was given a series of options for recording her activity including the use of a video 
camera, digital camera with video capability or using CamStudio™ an open source facility that records 
screen activity and audio. After trying all three methods she decided to use CamStudio™. This allows screen 
activity to be recorded but also allows the user to make a simultaneous voice over commentary. Jane made 
three recordings showing the process of downloading and uploading the materials and the difficulties that 
she encountered. This provided a useful insight into the problems of uploading and downloading content. 
These technical difficulties were reinforced from some data based on email questionnaires that had been 
given to a wider sample of users, some of whom had attempted to try to modify or upload material. Although 
Jane represented a potential content producer, as opposed to learner, it demonstrated the successful 
application of a remote monitoring technique, although admittedly she had a certain level of technical 
expertise in that she could handle the installation and running of CamStudio™. After the exercise she was 
further interviewed about her experience using FlashMeeting. The greatest benefits of this research were in 
terms of feedback to the team to help in the development of OpenLearn (as a form of action research) rather 
than in exploring theoretical and academic issues. However, in terms of the personal development of team 
members it helps toward developing ways and thinking about issues of monitoring remote experience. 

Other possible remote monitoring tools exist and in praxis we will experiment with various techniques to find 
those that work best. 

3. Discussion 

OpenLearn represents one of the largest developments within the open content community and presents a 
challenge for research. This challenge exists within four strands; teaching with OpenLearn, the users 
experience, project development, and sustainability. In this paper we have indicated and discussed the use 
of action research and activity theory as tools to enable us to think and understand the dynamics of a large 
educational initiative. Action research can potentially allow reflection, action and change within such a 
project. Activity Theory represents a tool for recognising areas for action and change and communicating 
issues to the project team. It also allows us as researchers to inwardly analyse our own behaviour and help 
in our personal and professional development. 

As researchers there is the need to disseminate internally and externally the research findings to inform 
change. The development of communication tools and the novel use of technology to do this is considered 
an evolutionary process, one of trial and error, experiment and change. Providing useful research networks 
and integrating with others is important in the social construction of knowledge and understanding about 
open educational resources. How to use tools such as videoconferencing (FlashMeeting), blogs, and 
dedicated website space effectively is a challenge and an important part of our own iterative process of 
development within an action research framework. Reflecting on our own research practice can be 
considered a meta-research process. 

Some of the research challenges of finding out about users’ experience have been illustrated. Possible 
technological tools that can help in this process have been discussed. A consideration of the use of tools as 
appropriated by individuals is a characteristic of the socio-technical view of action research. We have 
considered researching OpenLearn in terms of a number of different perspectives and themes. A three level 
approach has been presented. At one level OpenLearn can be viewed using activity theory to shape various 
perspectives and then examine intra-nodal and extra-nodal contradictions between the perspectives. At the 
level of the community of researchers there has been a consideration of the sharing and dissemination of 
knowledge and the tools that can aid this process. At the third level there has been a consideration of the 
challenge of developing research tools using the iterative processes of action research where, as in 
Engestrom’s (2005) ideas of expansion, people, tools and community are in a constant state of change. 
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